NPS-GSBPP-09-022 


NAVAL 

POSTGRADUATE 

SCHOOL 

MONTEREY,  CALIFORNIA 


Interorganizational  Collaborative  Capacity: 
Development  of  a  Database  to  Refine  Instrumentation 
and  Explore  Patterns 

24  August  2009 
by 

Erik  Jansen,  Senior  Lecturer 
Dr.  Susan  Page  Hocevar,  Associate  Professor 
Dr.  Rene  G.  Rendon,  Associate  Professor,  and 
Dr.  Gail  Fann  Thomas,  Associate  Professor 

Naval  Postgraduate  School 


Approved  for  public  release,  distribution  is  unlimited. 
Prepared  for:  Naval  Postgraduate  School,  Monterey,  California  93943 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


Naval  Postgraduate  School 
Monterey,  California 


Daniel  T.  Oliver  Leonard  A.  Ferrari 

President  Provost 


The  Acquisition  Program,  Graduate  School  of  Business  &  Public  Policy,  Naval 
Postgraduate  School  supported  the  funding  of  the  research  presented  herein. 
Reproduction  of  all  or  part  of  this  report  is  authorized. 

The  report  was  prepared  by: 


Erik  Jansen,  Senior  Lecturer 

Graduate  School  of  Operations  &  Information  Sciences 


Susan  Page  Hocevar,  Associate  Professor 
Graduate  School  of  Business  &  Public  Policy 


Rene  G.  Rendon,  Associate  Professor 
Graduate  School  of  Business  &  Public  Policy 


Dr.  Gail  Fann  Thomas,  Associate  Professor 
Graduate  School  of  Business  &  Public  Policy 

Reviewed  by: 


William  R.  Gates,  Ph.D. 

Dean,  Graduate  School  of  Business  &  Public  Policy 

Released  by: 


Karl  van  Bibber,  Ph.D. 

Vice  President  and  Dean  of  Research 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


REPORT  DOCUMENTATION  PAGE 


Form  approved 
OMB  No  0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources, 
gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this 
collection  of  information,  including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  information  Operations  and  Reports,  1215  Jefferson 
Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302,  and  to  the  Office  of  Management  and  Budget,  Paperwork  Reduction  Project  (0704-0188),  Washington,  DC  20503. 

I.  AGENCY  USE  ONLY  (Leave  blank)  2.  REPORT  DATE  3.  REPORT  TYPE  AND  DATES  COVERED 

24  August  2009  1  October  2007  through  30  September  2008 

4.  TITLE  AND  SUBTITLE  5.  FUNDING 

Interorganizational  Collaborative  Capacity:  Development  of  a  Database  to 
Refine  Instrumentation  and  Explore  Patterns 

6.  AUTHOR  (S) 

Erik  Jansen,  Susan  Page  Hocevar,  Rene  G.  Rendon,  and  Gail  Fann  Thomas _ 

7.  PERFORMING  ORGANIZATION  NAME  (S)  AND  ADDRESS  (ES)  8.  PERFORMING  ORGANIZATION  REPORT 

NAVAL  POSTGRADUATE  SCHOOL  NUMBER 

GRADUATE  SCHOOL  OF  BUSINESS  AND  PUBLIC  POLICY  NPS-GSBPP-09-022 

555  DYER  ROAD 

_ MONTEREY,  CA  93943-5103 _ 

9.  SPONSORING/MONITORING  AGENCY  NAME  (S)  AND  ADDRESS  (ES)  10.  SPONSORING/MONITORING 

AGENCY  REPORT  NUMBER 

II.  SUPPLEMENTARY  NOTES 


12a.  DISTRIBUTION/AVAILABILITY  STATEMENT  12b.  DISTRIBUTION  CODE 

Approved  for  public  release;  distribution  is  unlimited 

13.  ABSTRACT  (Maximum  200  words.) 

Interorganizational  collaborative  capacity  (ICC)  is  the  capability  of  organizations  (or  a  set  of  organizations)  to  enter 
into,  develop,  and  sustain  interorganizational  systems  in  pursuit  of  collective  outcomes.  The  objectives  of  the  ICC  research 
program  are  (1)  to  understand  the  success  factors  that  lead  to  and  the  barriers  that  interfere  with  ICC;  (2)  to  construct 
diagnostic  methods  and  tools  to  assess  these  factors;  and  (3)  to  develop  methods  that  contribute  to  the  development  of  ICC 
in  and  among  agencies  and  organizations. 

The  research  literature  indicates  that  a  major  barrier  blocking  progress  in  understanding  ICC  is  the  absence  of 
reliable,  valid  measures  for  the  construct.  This  study  addresses  this  problem.  It  presents  the  results  of  ICC  scale 
development  using  samples  of  public  sector,  defense  and  security  professionals  from  two  areas:  Homeland  Defense  and 
Security  and  Defense  Acquisition  and  Contracting.  The  research  presents  scales  that  have  very  good  to  excellent  internal- 
consistency  reliability  and  convergent  validity.  The  report  then  applies  the  survey  to  create  a  profile  and  do  a  summary 
assessment  of  a  major  DoD  Acquisition  and  Contracting  organization’s  ICC.  The  survey  factors  are  integrated  into  our  ICC 
open  systems  model.  The  value  of  survey  results  in  developing  an  organization’s  (or  an  organizational  set’s)  current  ICC  is 
discussed,  as  are  future  research  directions. 

14.  SUBJECT  TERMS  15.  NUMBER  OF 

Interorganizational  collaboration,  interagency  collaboration,  collaborative  capacity  pages 

_71 _ 

16.  PRICE  CODE 

17.  SECURITY  CLASSIFICATION  OF  18.  SECURITY  CLASSIFICATION  OF  19.  SECURITY  CLASSIFICATION  OF  20.  LIMITATION  OF 

REPORT:  UNCLASSIFIED _  THIS  PAGE:  UNCLASSIFIED _  ABSTRACT:  UNCLASSIFIED _  ABSTRACT:  UU _ 

NSN  7540-01-280-5800  Standard  Form  298  (Rev.  2-89) 

Prescribed  by  ANSI  Std  239-18 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


Abstract 


Interorganizational  collaborative  capacity  (ICC)  is  the  capability  of 
organizations  (or  a  set  of  organizations)  to  enter  into,  develop,  and  sustain 
interorganizational  systems  in  pursuit  of  collective  outcomes.  The  objectives  of  the 
ICC  research  program  are  (1 )  to  understand  the  success  factors  that  lead  to  and  the 
barriers  that  interfere  with  ICC;  (2)  to  construct  diagnostic  methods  and  tools  to 
assess  these  factors;  and  (3)  to  develop  methods  that  contribute  to  the  development 
of  ICC  in  and  among  agencies  and  organizations. 

The  research  literature  indicates  that  a  major  barrier  blocking  progress  in 
understanding  ICC  is  the  absence  of  reliable,  valid  measures  for  the  construct.  This 
study  addresses  this  problem.  It  presents  the  results  of  ICC  scale  development 
using  samples  of  public  sector,  defense  and  security  professionals  from  two  areas: 
Homeland  Defense  and  Security  and  Defense  Acquisition  and  Contracting.  The 
research  presents  scales  that  have  very  good  to  excellent  internal-consistency 
reliability  and  convergent  validity.  The  report  then  applies  the  survey  to  create  a 
profile  and  do  a  summary  assessment  of  a  major  DoD  Acquisition  and  Contracting 
organization’s  ICC.  The  survey  factors  are  integrated  into  our  ICC  open  systems 
model.  The  value  of  survey  results  in  developing  an  organization’s  (or  an 
organizational  set’s)  current  ICC  is  discussed,  as  are  future  research  directions. 

Keywords:  Interorganizational  collaboration,  interagency  collaboration, 
collaborative  capacity 
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Collaboration  and  the  Acquisition  Context 


Interorganizational  collaboration  comprises  a  system  of  processes  by  which 
organizations  work  together  to  accomplish  common  or  complementary  goals  and 
objectives  or  a  common  mission.  Collaborations  range  from  (a)  close  partnerships  in 
which  employees  from  two  or  more  organizations  must  work  interdependently  to 
accomplish  goals  and  objectives  to  (b)  low-level  cooperation  in  which  two  or  more 
organizations  share  information  so  that  each  can  be  more  efficient  in  accomplishing  its 
own,  relatively  independent  goals.1  Collaboration  is  often  used  synonymously  with 
partnering  and  is  manifest  when  organizations  form  alliances.  We  formally  define 
interorganizational  collaborative  capacity  as  the  capability  of  organizations  (or  a  set  of 
organizations)  to  enter  into,  develop,  and  sustain  interorganizational  systems  in  pursuit 
of  collective  outcomes. 

Federal  Acquisition  Reform  has  consistently  called  for  improved  collaboration 
among  participating  acquisition  agencies  as  well  as  between  the  Department  of 
Defense  (DoD)  and  defense  contractors.  DoD  Directive  5000.1  (Undersecretary  for 
Defense,  Acquisition  Technology  &  Logistics,  2003,  paragraph  El .)  states  that  teaming 
among  warfighters,  users,  developers,  acquirers,  technologists,  testers,  budgeters,  and 
sustainers  is  required  during  the  capability  needs  definition  phase  of  the  acquisition 
lifecycle.  Furthermore,  a  Defense  Acquisition  Performance  Assessment  (DAPA) 
recommends  improved  collaboration  among  acquisition  organizations  and  between  DoD 
and  industry.  The  use  of  integrated  product  teams  (IPTs),  partnering  relationships,  and 
alpha  contracting  processes  are  examples  of  collaborative  processes  being  used  in 
some  commands. 

Collaboration  among  organizations  has  clear  benefits,  including  reduced 
litigation,  cost  savings  through  the  transfer  of  smart  practices,  better  decision  making  as 
a  result  of  advice  and  shared  information,  enhanced  capacity  for  dispersed  units  to  act 


1  For  those  familiar  with  J.  D.  Thompson’s  (1967)  ideas,  the  distinction  reflects  what  he  characterizes  as 
low-level,  “pooled  interdependence”  versus  high  level,  “reciprocal  interdependence.” 
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collectively,  and  innovation  resulting  from  the  cross-pollination  of  ideas  and 
recombination  of  scarce  resources  (Hansen  &  Nohria,  2004;  Mankin,  Cohen  & 
Fitzgerald,  2004).  However,  organizations  sometimes  fall  short  when  they  attempt  to 
develop  effective  collaborative  relationships.  Documented  barriers  include  diverse  and 
conflicting  missions,  goals,  and  incentives;  entrenched  histories  of  distrust;  leaders  who 
do  not  actively  support  collaborative  efforts;  and  inadequate  systems  and  structures  (US 
Government  Accountability  Office,  2005).  Thus  a  practical  as  well  as  a  theoretical  need 
exists  for  understanding  the  factors  that  account  for  successful  versus  unsuccessful 
collaboration,  and  for  understanding  the  capacities  or  capabilities  of  organizational 
collaboration. 
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The  Interorganizational  Collaborative  Capacity  (ICC) 
Research  Project 


Since  the  summer  of  2002,  our  research  has  moved  through  various  phases, 
including  reviews  of  the  literature,  inductively  based  empirical  research,  and  survey 
development.  The  focus  of  the  research  presented  in  this  report  is  on  instrument 
refinement  and  the  exploration  of  data  patterns  in  several  DoD  samples.  We  also  use 
data  from  a  major  DoD  Contracting  Administration  organization  to  briefly  illustrate  the 
use  of  the  measurement  tool  for  organizational  assessment,  which  can  serve  as  a  basis 
for  survey-guided  organizational  development. 

Inductively  Generating  a  Model  of  Interorganizational 
Collaborative  Capacity 

In  2002/2003,  supported  by  funding  from  the  Naval  Postgraduate  School’s 
Center  for  Homeland  Defense  and  Security,  we  conducted  an  educational  workshop 
and  collected  additional  data  from  mid-level  and  senior  Homeland  Security  and  Defense 
professionals  to  inductively  derive  a  conceptual  model  of  collaborative  capacity.  Our 
focus  was  on  collaborative  capacity  in  a  planning  or  pre-crisis  context  rather  than  in  a 
post-response  context.  This  empirical  research,  combined  with  a  review  of  the 
literature,  allowed  us  to  identify  enablers  and  barriers  to  collaboration.  This  inductive 
study  is  described  in  a  previous  publication  (Hocevar,  Thomas,  &  Jansen,  2006);  the 
major  results  of  the  thematic  analysis  are  presented  in  Table  1 . 
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Table  1.  Thematic  Factors  that  Contribute  to  Success  or  Serve  as  Barriers  to 

Interorganizational  Collaboration 


Organization 

Domain 

“Success”  factors1 

“Barrier”  factors1 

Purpose  and 

■  “Felt  need”  to  collaborate 

■  Divergent  goals 

Strategy 

■  Common  goal  or 

■  Focus  on  local  organization 

recognized 

over  cross-agency  (e.g., 

interdependence1 

regional)  concerns 

■  Adaptable  to  interests  of 

■  Lack  of  goal  clarity 

other  organizations 

■  Inadequate  resources 

■  Not  adaptable  to  interests  of 
other  organizations 

Structure 

■  Sufficient  authority  of 

■  Impeding  rules  or  policies 

participants 

■  Inadequate  authority  of 
participants 

■  Lack  of  accountability 

■  Lack  of  formal  roles  or 
procedures  for  managing 
collaboration 

Lateral  Processes 

■  Social  capital  (i.e. , 

■  Lack  of  familiarity  with 

interpersonal  networks) 

other  organizations 

■  Effective  communication 

■  Inadequate  communication 

and  information  exchange 

and  information  sharing 

■  Formalized  coordination 
committee  or  liaison  roles 

■  Technical  interoperability 

(distrust) 

Incentives  and 

■  Collaboration  as  a 

■  Competition  for  resources 

Reward  Systems 

prerequisite  for  funding  or 

■  Territoriality 

resources 

■  Organization-level  distrust 

■  Leadership  support  and 

■  Lack  of  mutual  respect 

commitment 

■  Absence  of  competitive 
rivalries 

■  Acknowledged  benefits  of 
collaboration  (e.g.,  shared 
resources) 

■  Apathy 

People: 

■  Appreciation  of  others’ 

■  Lack  of  competency 

Individual 

perspectives 

■  Arrogance,  hostility, 

Collaborative 

■  Competencies  for 

animosity 

Capacity 

1  _  ■  _  ■  . 

collaboration 

■  Trust 

■  Commitment  and  motivation 

1  Items  in  bold  were  identified  by  at  least  25%  of  the  study  participants. 


(Modified  from  Hocevar,  Thomas,  &  Jansen,  2006) 
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The  Collaborative  Capacity  Model.  The  themes  identified  in  Table  1  seemed 
to  best  fit  and  be  mapped  onto  the  domains  of  Galbraith’s  (2002)  open  systems  model 
of  organizations.  The  ICC  model  in  Figure  1  (Hocevar,  Thomas,  &  Jansen,  2006) 
represents  an  image  of  a  simplest  case  of  interorganizational  collaboration  involving 
only  two  organizations — A  and  B — in  a  shared  problem  space.  The  figure’s  double¬ 
sided,  yellow  arrows,  which  lead  from  each  organization  to  “goals  and  objectives,”  are 
meant  to  indicate  that  each  organization  is  oriented  toward  its  own  goals  and  objectives 
according  to  its  special  interests  and  responsibilities.  Each  organization’s  goal-directed 
behavior  affects  the  common  problem  space  and  thus  affects  the  other  organization, 
sometimes  dramatically.  They  operate  in  an  interdependent  relationship  in  which  some 
degree  of  cooperation  and  collaboration  can  improve  their  efficiency  and  effectiveness.2 

Each  organization  can  be  conceptualized  as  a  complex  adaptive  system  with  five 
subsystem  domains.  These  are  strategy  and  purpose,  organizational  structure,  reward 
systems,  people,  and  lateral  processes,  which  are  represented  by  the  points  of  the 
pentagon  in  Figure  1  (of.  Galbraith,  2002).  As  with  other  open  systems  models,  the  ICC 
model  emphasizes  that  the  efficiency  and  effectiveness  of  each  organization  depends 
on  the  congruence  or  fit  (i.e.,  fitness)  of  its  subsystems.  For  example,  efficiency  is 
increased  when  an  organization’s  incentives  and  reward  systems  are  congruent  with  its 
strategic  goals,  structure  of  authority  and  responsibilities.  The  single-sided,  blue  arrows 
indicate  the  self-reinforcing  dynamics  by  which  collaborative  capacity  is  generated 
through  the  interplay  of  the  five  subsystem  domains. 


2  This  does  not  mean  there  are  not  conflicts  or  competitive  aspects  to  their  interactions.  Ray  Norda  re¬ 
introduced  the  term  “coopetition”  to  describe  the  complex  dynamics  in  which  organizations  could  be  allies 
in  one  problem  space  and  competitors  in  a  different  problem  space  (Fisher,  1992). 


-5- 


Interorganizational  Problem  Space 


Figure  1.  The  Interorganizational  Collaborative  Capacity  Model 
(Hocevar,  Thomas,  &  Jansen,  2006) 


The  role  of  the  interagency  team  is  to  generate  collaborative  capacity  in  both 
organizations.  But  that  team  must  also  develop  its  own  collaborative  capacity;  again, 
the  single-sided,  blue  arrows  suggest  these  dynamics.  Of  course,  as  there  may  be 
more  than  two  organizations  collaborating,  there  also  may  be  more  than  a  single 
interagency  task  force.  It  is  common  to  have  multiple  task  forces,  special  teams,  or 
tiger  teams  focusing  on  and  aligning  specific  subsystem  domains  (e.g.,  policies  and 
procedures  for  sharing  information). 

The  Measurement  Problem.  The  literature  review  (cf.  Bardach,  1998)  indicated 
that  a  major  barrier  blocking  progress  in  understanding  ICC  is  the  absence  of  reliable, 
valid  measures  for  the  construct.  Having  developed  an  overarching  model  with 
identified  thematic  factors,  we  undertook  the  present  research  to  confront  the  question, 
“How  can  we  measure  or  assess  interorganizational  collaborative  capacity?”  We  wrote: 
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Collaborative  capacity  is  an  intuitively  appealing  construct  but  currently 
lacks  clear  operationalization.  This  deficiency  is  problematic  for  leaders 
and  practitioners  [...]  who  want  to  identify  the  collaborative  capacity  of 
their  agencies.  The  absence  of  measurement  models  also  is  problematic 
for  the  advancement  of  the  social  science  of  interagency  collaboration. 

The  factors  in  [our  studies...]  were  inductively  generated  in  the  hope  of 
identifying  dimensions  that  need  to  be  measured  to  operationalize  the 
construct.  Thus,  the  next  challenge  researchers  must  face  is  how  to 
diagnose  or  audit  the  collaborative  capacity  of  organizations  that  are 
expected  to  be  in  effective  collaborative  relationships.  (Hocevar,  Thomas, 

&  Jansen,  2006,  p.  273) 

Measuring  Interorganizational  Collaborative  Capacity 

From  2006  through  2008,  in  research  funded  partly  by  the  Acquisition  Research 
Program  (see  Appendix  A),  we  began  developing  a  systematic  means  of  assessing  an 
organization’s  (or  an  organizational  set’s)  collaborative  capacity.  We  continued  to  work 
in  the  context  of  Homeland  Security  and  Defense  but  expanded  our  scope  to  include  the 
context  of  Defense  Acquisition  and  Contracting.3  A  diagnostic  process  was  developed 
that  included  survey  and  interview  questions.  Our  primary  focus  was  to  develop  a 
quantitative  instrument — a  survey — that  could  be  administered  through  the  Internet  to 
assess  individuals’  perceptions  of  their  organization’s  ICC.  One  criterion  for  generating 
items  was  their  potential  to  suggest  courses  of  action  for  leaders  and  managers  facing 
the  challenges  of  developing  ICC.  Another  criterion  was  to  cover  important  factors 
within  each  of  the  five  major  subsystem  domains  of  the  ICC  model. 

This  resulted  in  the  creation  of  an  item  bank  of  survey  items,  as  well  as  interview 
questions  that  went  through  a  process  of  judgment  involving  the  research  team, 
graduate  students  and  subject  matter  experts  (SMEs)  to  select  the  most  promising 
items.  Most  of  the  survey’s  items  assess  factors  identified  in  our  research  and  the 
literature.  Additional  items  assess  organizational  level  demographics  (e.g.,  years  of 
experience  with  interagency  partnerships,  number  of  employees,  and  geographic 


3  In  2005,  Professor  Rene  Rendon  was  added  to  our  ICC  research  team  for  his  expertise  and  experience 
in  acquisition  and  contracting. 
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proximity  of  organizational  partners)  or  respondent  demographics  (e.g.,  organizational 
tenure).  These  efforts  are  described  in  Thomas,  Hocevar,  Jansen,  and  Rendon  (2008). 


This  brings  us  to  the  current  report,  in  which,  after  nearly  a  dozen  different 
administrations  of  the  instrument,  most  of  which  involved  feedback  sessions  with 
respondents,  we  present  scales  for  a  survey  instrument  to  assess  ICC.  We  then 
illustrate  the  survey’s  promise  by  creating  and  discussing  an  ICC  profile  for  two  units  of 
a  major  DoD  Acquisition  and  Contracting  organization. 
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Instrument  Refinement  and  Construct  Validation 


In  this  section,  the  focus  of  the  research  is  on  (a)  developing  scales  with  high 
reliability  as  operationalized  by  Cronbach’s  coefficient  alpha  and,  simultaneously,  (b) 
validating  the  construct  of  collaborative  capacity.  Items  are  the  focus  of  the  analysis, 
and  respondents  are  combined  to  discern  the  relationships  among  items.  The  purpose 
here  is  not  to  use  items  to  understand  the  behavior  of  people  but  to  use  the  responses 
of  people  to  understand  and  validate  the  behavior  of  items.  In  subsequent  sections,  the 
focus  shifts  to  using  the  resulting  scales  to  describe  organizations  and  organization 
sets. 

Samples  for  Instrument  Validation 

The  survey  respondents  in  the  present  study  are  individuals  representing  two 
communities:  Homeland  Defense  and  Security  and  Acquisition  and  Contracting.  The 
Homeland  Defense  and  Security  sample  is  composed  of  145  students  in  six  classes  in  a 
master’s  degree  program  in  Homeland  Security  at  the  Naval  Postgraduate  School. 

They  are  experienced  civilians  or  military  officers  working  for  civilian,  government  or 
military  organizations  from  around  the  United  States  who  have  on-going  Homeland 
Security  responsibilities.  Illustrative  organizations  (and  positions)  include: 
USNORTHCOM,  US  Coast  Guard,  (mid-  to  senior-level  officers),  the  Center  for  Disease 
Control,  Offices  of  Emergency  Management,  the  FBI,  municipal  police  and  fire 
departments,  and  utilities  (Directors,  Chiefs,  Captains). 

The  Acquisition  and  Contracting  sample  includes  49  DoD  managers,  specifically 
program  managers,  technical/engineering  managers,  and  contract  managers.  They 
were  in  three  classes  pursuing  an  MS  in  Program  Management  (MSPM)  though 
distance  learning  and  employed  full-time  as  members  of  the  DoD  acquisition  workforce. 
These  students  are  experienced  DoD  acquisition  managers,  many  of  whom  had  already 
achieved  Level  II  of  the  Defense  Acquisition  Workforce  Improvement  Act  (DAWIA) 
certification  program.  The  students  were  located  across  the  United  States  at  some  of 
the  DoD's  major  acquisition  and  procurement  centers. 
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The  Acquisition  and  Contracting  sample  also  includes  79  federal  employees  from 
a  DoD  Research,  Development  and  Fielding  Organization  that  is  responsible  for 
engineering,  technology,  research,  development,  and  fielding  products  primarily  for 
individual  military  personnel.  The  organization  comprises  three  interdependent 
organizations  from  each  of  the  military  services  and  works  with  academic  organizations, 
research  hospitals,  businesses,  and  other  government  agencies. 

The  Acquisition  and  Contracting  sample  also  includes  46  employees  from  a 
major  DoD  Contracting  Administration  Organization  who  hold  to  the  mission  of 
improving  integration  of  acquisition  processes  between  DoD  clients  and  contractors, 
improving  cost  efficiency,  increasing  process  innovations,  and  ensuring  compliance  to 
standards  and  regulations  in  federal  contracts. 

Methods 

In  developing  scales  possessing  internal  consistency  reliability  and  convergent 
validity,  it  is  important  to  have  (a)  a  sufficient  number  of  respondents  and  (b) 
respondents  who  have  a  range  of  perceptions  regarding  their  organization’s  ICC  (i.e. , 
variance  in  the  scale  values).  The  item  results  below  reveal  considerable  variance 
among  the  respondents  as  they  judge  the  ICC  of  their  respective  organizations. 

The  scales  were  developed  in  an  iterative  process  that  involved  collecting  data, 
giving  respondents  feedback  on  their  results,  and  listening  to  their  interpretation  of  the 
results  for  specific  items  and  the  resulting  scales.  In  some  cases  items  that  appeared  to 
be  statistically  useful  were  rejected  because  of  feedback  in  the  educational  workshops 
with  our  students,  who  also  were  subject  matter  experts  (SME).  In  other  cases,  items 
were  rejected  because  they  failed  to  meet  statistical  hurdles.  Sometimes,  new  items 
were  suggested  or  items  were  revised  based  on  feedback.  One  result  of  this  iterative 
process  is  the  smaller  sample  sizes  available  for  some  items  and  their  associated 
scales. 


The  sample  sizes  are  not  yet  large  enough  to  conduct  complete  multivariate 
analyses  (e.g.,  exploratory  or  confirmatory  factor  analyses  that  include  all  the  items). 
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We  have  sufficient  respondents  to  run  a  series  of  exploratory  factor  analyses  (Principal 
Components  Analysis,  varimax  solutions,  orthogonal  rotations)  on  subsets  of  items. 
This,  along  with  the  coefficient  alpha  analyses  and  the  analyses  based  on  content  or 
face  validity,  provides  the  primary  justification  for  the  scales.  We  expect  to  have 
sufficient  sample  size  within  the  2009  academic  year  to  conduct  the  multivariate 
analyses  noted  above. 

Results:  Survey  Items  and  Scales 

In  this  section,  we  discuss  the  factors  that  operationally  define  the  ICC  construct. 
This  section  focuses  on  the  items  that  assess  collaborative  capacity  as  opposed  to 
demographic  items.  Figure  2  shows  an  example  of  one  item  and  the  response  scale. 

All  items  used  a  6-point  Likert-type  scale  ranging  from  Strongly  Disagree  to  Strongly 
Agree.  Thus,  3.5  is  the  scale  midpoint.  Sample  means  higher  than  3.5  indicate 
agreement,  and  a  sample  mean  of  6  would  indicate  unanimous  agreement.  In  total,  we 
created  12  scales.  The  12  scales  are  Need  to  Collaborate,  Strategic  Collaboration, 
Resource  Investment  in  Collaboration,  Structural  Flexibility,  Reward  Systems,  Metrics 
for  Collaboration,  Information  Sharing,  Collaborative  Learning,  Social  Capital,  Individual 
Collaborative  Capacity,  Barriers  to  Collaboration,  and  Interagency  Teams.  We  present 
each  scale  below  with  its  respective  component  items  and  the  mean  values  calculated 
from  the  total  combined  sample. 


la  my  organization,  people  take  the  initiative  to  luiild  relationships 

with  their  counterparts  in  other  organizations. 

1  2  3  4  5 

6 

Strongly 

Strongly 

Disagree 

Agree 

Figure  2.  Sample  Item  with  the  6-point  Likert-Type  Scale 


Need  to  Collaborate.  Hocevar,  Thomas,  and  Jansen  (2006)  used  the  term  “felt 
need”  to  refer  to  a  theme  that  emerged  in  their  inductive,  qualitative  research  as  well  as 
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in  their  literature  review.  “Felt  need”  was  taken  from  the  organizational  change  literature 
(Jick,  2002)  in  which  the  construct  is  used  to  describe  the  strong  motivational  energy 
and  effort  needed  to  overcome  the  inertia  of  the  status  quo  and  generate  change  in 
organizational  structures,  processes  and  behaviors.  The  change  literature  asserts  that 
a  felt  need  or  “sense  of  urgency”  (Kotter,  2008)  is  a  powerful  factor  that  motivates 
individuals  to  make  commitments  to  learning  new  skills  and  exploring  new  behaviors. 
The  Need  to  Collaborate  scale  presented  in  Table  2  has  very  good  reliability.  Both  the 
scale  and  the  item  means  are  greater  than  4.5,  indicating  that  this  factor  is  a  high 
priority,  with  understood  benefits  for  accomplishing  the  organization’s  mission. 


Table  2.  Means,  Standard  Deviations,  and  Coefficient  Alpha  for  the  Need  to 

Collaborate  Items  and  Scale 


Need  to  Collaborate  Items 

Item 

Item 

Mean 

Standard 

Deviation 

Interorganizational  collaboration  is  a  high  priority  for  my 

4.8 

1.4 

organization. 

My  organization  recognizes  the  importance  of  working  with 

4.8 

1.3 

other  agencies  to  achieve  its  mission. 

People  in  my  organization  understand  the  benefits  of 

4.5 

1.2 

collaborating  with  other  organizations. 

Need  to  Collaborate  Scale  Statistics 

Mean  Standard  Deviation  Sample  Size 

Coefficient  Alpha 

4.7  1.3  307 

81 
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Strategic  Collaboration.  Another  important  theme  that  emerged  for  successful 
collaboration  in  our  inductive  research  was  having  “a  common  goal  or  recognized 
interdependence.”  The  items  in  Table  3  are  congruent  with  this  factor.  They  emphasize 
establishing  and  addressing  goals  for  collaboration  and  considering  the  interest  of  other 
agencies  in  planning.  They  also  focus  on  the  role  of  leadership  in  addressing 
interorganizational  goals  and  conferring  with  the  leaders  of  other  organizations.  This 
emphasis  on  goals,  planning  and  leadership  led  to  this  factor  being  placed  in  the 
strategy  and  purpose  domain.  The  coefficient  alpha  of  internal  consistency  is  very  good 
to  excellent. 

Table  3.  Means,  Standard  Deviations,  and  Coefficient  Alpha  for  the  Strategic 

Collaboration  Items  and  Scale 


Item 

Item 

Strategic  Collaboration  Items 

Mean 

Standard 

Deviation 

We  have  clearly  established  goals  for  interorganizational 
collaboration. 

4.1 

1.4 

The  leaders  of  my  organization  emphasize  the  importance  of 
collaboration. 

4.6 

1.4 

My  organization  is  willing  to  address  interorganizational  goals. 

4.6 

1.2 

My  organization's  leaders  meet  and  confer  with  the  leaders  of 
other  organizations  about  mutual  collaboration. 

4.2 

1.4 

My  organization  considers  the  interests  of  other  agencies  in  its 
planning. 

3.9 

1.4 

Strategic  Collaboration  Scale  Statistics 

Mean  Standard  Deviation  Sample  Size 

Coefficient  Alpha 

4.3  1.4  251 

.85 
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Resource  Investment  in  Collaboration.  Resource  Investment  in  Collaboration 
emerged  as  a  scale  in  this  research;  it  also  emerged  as  a  theme  in  our  inductive 
research.  The  items  in  Table  4  focus  on  investing,  committing  or  assigning  budget, 
resources  and  personnel  to  interorganizational  collaboration.  The  commitment  of  such 
resources  is  viewed  as  best  fitting  in  the  domain  of  strategy  and  purpose  since  the 
strategic  apex  is  where  resource  decisions  typically  are  made.  The  reliability  of  the 
scale  is  very  good  to  excellent. 

Table  4.  Means,  Standard  Deviations,  and  Coefficient  Alpha  for  the  Resource 
Investment  in  Collaboration  Items  and  Scale 


Resource  Investment  in  Collaboration  Items  Item  Item 

Mean  Standard 

_ Deviation 

My  organization  has  committed  adequate  time,  budget,  and  3.8  1 .4 

personnel  to  interorganizational  collaboration.1 

My  organization  is  willing  to  invest  resources  to  accomplish  cross-  4.1  1.4 

agency  goals. 

My  organization  has  assigned  adequate  personnel  to  the  work  3.3  1 .4 

required  for  effective  interorganizational  collaboration. _ 

Resource  Investment  in  Collaboration  Scale  Statistics 

_ Mean _ Standard  Deviation _ Sample  Size _ Coefficient  Alpha 

3.7  1.4  227  .88 


This  item  has  been  revised  for  current  usage.  It  now  reads:  “My  organization  has  committed  adequate 
budget  and  resources  to  interorganizational  collaboration.”  This  statement  removes  the  overlap  between 
the  first  and  third  items  in  this  Table. 
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Structural  Flexibility.  In  the  inductive  research,  structural  themes  emerged 
somewhat  less  frequently  than  other  themes,  and,  as  Table  1  reveals,  they  were 
mentioned  more  often  as  barriers  than  success  factors.  Deming’s  (1982)  approaches  to 
quality,  profound  learning  and  continuous  quality  management  emphasized  that  the 
importance  of  the  larger  organizational  system  and  structure  on  individual  behaviors  is 
generally  underestimated.  Thus,  we  might  expect  that  structural  factors  to  be  under¬ 
represented  in  the  themes  people  generate,  compared  to  themes  involving  personal 
motivations  and  skills,  dysfunctional  incentives,  goals,  and  communication. 

We  were  able  to  create  a  scale  relating  to  the  structure  subsystem  domain  that 
assesses  Structural  Flexibility.  These  items  measure  the  degree  to  which  respondents 
perceive  that  the  organization  is  flexible  and  responsive,  quickly  forming  and  modifying 
policies,  processes,  procedures,  and  partnerships.  Table  5  presents  the  item  and  scale 
statistics  and  shows  good  scale  reliability. 

Table  5.  Means,  Standard  Deviations,  and  Coefficient  Alpha  for  the  Structural 

Flexibility  Items  and  Scale 


Structural  Flexibility  Items 

Item 

Mean 

Item 

Standard 

Deviation 

My  organization  invests  significant  time  and  energy  to  deconflict 
existing  policies  and  processes  that  impede  collaboration. 

3.4 

1.3 

My  organization  is  flexible  in  adapting  its  procedures  to  better  fit 
with  those  organizations  with  which  we  work  or  might  work. 

4.2 

1.3 

My  organization  is  responsive  to  the  requirements  of  other 
organizations  with  which  we  work. 

4.4 

1.1 

My  organization  can  quickly  form  or  modify  partnerships  as 
requirements  change. 

4.5 

1.3 

Structural  Flexibility  Scale  Statistics 

Mean 

Standard  Deviation 

Sample  Size 

Coefficient  Alpha 

4.1 

1.2 

135 

.78 

-  15  - 


Reward  Systems.  The  importance  of  Reward  Systems  for  activities  related  to 
ICC  emerges  strongly  in  the  inductive  research,  and  the  items  in  Table  6  written  to 
assess  this  factor  have  very  good  to  excellent  reliability.  These  items  assess  the 
individuals’  perceptions  of  the  consequences  of  their  behavior  in  terms  of  their  own 
personal  payoffs.  The  items  assess  the  degree  to  which  collaborative  work  and 
activities  and  collaborative  talents  result  in  rewards,  career  advancement,  and 
promotion. 


Table  6.  Means,  Standard  Deviations,  and  Coefficient  Alpha  for  the  Reward 

Systems  Items  and  Scale 


Reward  Systems  Items 

Item 

Mean 

Item 

Standard 

Deviation 

Engaging  in  interagency  activities  at  work  is  important  to  career 
advancement  in  this  organization. 

3.7 

1.6 

My  organization  rewards  employees  for  investing  time  and  energy 
in  building  collaborative  relationships. 

3.3 

1.5 

My  organization  rewards  members  for  their  IA  collaborative 
activities. 

3.3 

1.4 

Collaborative  talents  and  achievements  are  considered  when 
people  are  reviewed  for  promotion. 

3.4 

1.6 

Reward  Systems  Scale  Statistics 

Mean 

Standard  Deviation 

Sample  Size 

Coefficient  Alpha 

3.4 

1.5 

268 

.86 
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Metrics  for  Collaboration.  Metrics  did  not  emerge  as  a  theme  in  the  inductive 
research,  but  the  two  questions  in  Table  7  combined  to  form  a  scale  as  the 
measurement  model  was  developed.  The  items  have  very  good  reliability  with  an  alpha 
coefficient  of  .83.  They  assess  the  degree  to  which  an  organization  has  identified  or 
established  measurement  criteria  and  performance  standards  to  assess  interagency 
collaboration  efforts. 

In  Figure  6,  Metrics  is  placed  between  the  strategy  and  purpose  domain  and  the 
reward  systems  domain.  On  the  one  hand,  metrics  operationalize  goals,  often  strategic 
goals,  and  they  may  serve  as  indicators  of  effectiveness.  On  the  other  hand,  metrics 
can  be  integrated  into  the  performance  appraisal  process  and  thus  become  part  of  the 
reward  system. 

Table  7.  Means,  Standard  Deviations,  and  Coefficient  Alpha  for  the  Metrics  for 

Collaboration  Items  and  Scale 


Metrics  for  Collaboration  Items 

Item 

Item 

Mean 

Standard 

Deviation 

My  organization  has  identified  measurement  criteria  to  evaluate 
interorganizational  efforts. 1 

3.0 

1.5 

My  organization  has  established  clear  performance  standards 
regarding  interorganizational  work. 1 

3.1 

1.4 

Metrics  for  Collaboration  Scale  Statistics 

Mean  Standard  Deviation  Sample  Size 

Coefficient  Alpha 

3.0  1.5  264 

1  i .  xi  ii.  i  i  .  .. _:x.  .  t  r\  £  .  _ .1  xi  ...  i .  m:  .  x  .. 

1  XI 

.83 

1  In  the  Homeland  Security  and  Defense  samples,  the  wording  “interagency”  was  used  rather  than 
“interorganizational.” 
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Information  Sharing.  “Information  sharing”  emerged  as  a  theme  in  the 
Hocevar,  Thomas,  and  Jansen  (2006)  research  in  the  domain  of  lateral  mechanisms  or 
lateral  processes.  Three  items  in  Table  8  make  up  an  Information  Sharing  scale  with 
very  good  reliability.  The  items  are  diverse;  they  refer  to  people,  the  organization,  and 
to  organizational  norms. 

As  we  noted  when  we  defined  collaboration,  some  work  relationships 
demonstrate  low  levels  of  interdependence  (i.e.,  pooled  interdependence)  in  which 
information  sharing  represents  all  that  is  required  of  organizational  allies.  Such 
organizations  are  able  to  accomplish  their  goals  and  objectives  independently  if 
information  sharing  is  effective;  more  developed  collaborative  relationships  (e.g., 
liaisons,  regular  task  force  meetings,  joint  exercises)  are  unnecessary. 

Table  8.  Means,  Standard  Deviations,  and  Coefficient  Alpha  for  the  Information 

Sharing  Items  and  Scale 


Information  Sharing  Items 

Item 

Mean 

Item 

Standard 

Deviation 

My  organization  has  strong  norms  that  encourage  sharing 
information  with  other  organizations. 1 

3.8 

1.5 

My  organization  provides  other  organizations  adequate  access  to 
information  we  have  that  is  relevant  to  their  work. 1 

4.1 

1.4 

People  in  my  organization  share  information  with  other 
organizations. 

4.4 

1.2 

Information  Sharing  Scale  Statistics 

Mean 

Standard  Deviation 

Sample  Size 

Coefficient  Alpha 

4.1 

1.4 

226 

.83 

1 1n  the  Homeland  Security  and  Defense  samples,  “agencies”  was  used  rather  than  “organizations.” 
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Collaborative  Learning.  When  organizations  face  problems  that  require 
teamwork,  either  because  of  sequential  or  reciprocal  interdependence,  learning  how  to 
work  with  organizational  partners  becomes  more  important.  The  three  items  in  Table  9 
form  a  scale  with  very  good  to  excellent  reliability.  The  items  assess  the  degree  to 
which  the  organization  commits  resources  to  training,  works  with  other  organizations  to 
identify  lessons  learned  and  develops  strong  norms  for  learning  from  other 
organizations.  They  assess  the  degree  to  which  the  organization  might  be  regarded  as 
a  collaborative  learning  organization.  We  judge  that  this  scale  belongs  in  the  lateral 
processes  domain  of  the  collaborative  capacity  model. 

Table  9.  Means,  Standard  Deviations,  and  Coefficient  Alpha  for  the  Collaborative 

Learning  Items  and  Scale 


Collaborative  Learning  Items 

Item 

Mean 

Item 

Standard 

Deviation 

My  organization  commits  adequate  human  and  financial  resources 
to  training  with  other  organizations. 1 

3.3 

1.4 

My  organization  has  strong  norms  for  learning  from  other 
organizations. 1 

My  organization  works  with  other  organizations1  to  identify  lessons 
learned  for  improved  collaboration. 

3.6 

1.4 

3.5 

1.3 

Collaborative  Learning  Scale  Statistics 

Mean 

Standard  Deviation 

Sample  Size 

Coefficient  Alpha 

3.5 

1.4 

225 

.85 

1 1n  the  Homeland  Security  and  Defense  samples,  “agencies”  was  used  rather  than  “organizations.” 
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Social  Capital.  “Social  capital”  and  “interpersonal  networks”  emerged  as  an 
important  theme  associated  with  success  in  Hocevar,  Thomas,  and  Jansen  (2006),  and 
the  two  items  in  Table  10  have  good  internal-consistency  reliability,  thus  forming  a  scale 
in  the  lateral  processes  domain.  They  assess  the  degree  to  which  organizational 
employees/members  take  the  initiative  to  build  relationships  and  know  who  to  contact  in 
other  organizations  or  agencies. 


Table  10.  Means,  Standard  Deviations,  and  Coefficient  Alpha  for  the  Social  Capital 

Items  and  Scale 


Social  Capital  Items 

Item 

Item 

Mean 

Standard 

Deviation 

Our  employees  know  who  to  contact  in  other  agencies  for 
information  or  decisions. 

4.1 

1.4 

Members  of  my  organization  take  the  initiative  to  build 
relationships  with  their  counterparts  in  other  organizations. 

4.3 

1.3 

Social  Capital  Scale  Statistics 

Mean  Standard  Deviation  Sample  Size 

Coefficient  Alpha 

4.2  1.3  307 

.79 
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Individual  Collaborative  Capacity.  Hocevar,  Thomas,  and  Jansen  (2006) 
identified  a  number  of  themes  describing  the  collaborative  capabilities  and  attitudes  of 
individuals  within  the  organization.  The  seven  items  in  Table  1 1  are  combined  into  an 
Individual  Collaborative  Capacity  scale  that  has  very  good  to  excellent  reliability.  These 
include  items  that  focus  on  skills,  capabilities  and  expertise,  understanding  and 
knowledge  of  other  organizations  work  and  perspective,  willingness  to  engage  in  shared 
decision-making,  and  seeking  input  from  the  other  organization. 

Table  11.  Means,  Standard  Deviations,  and  Coefficient  Alpha  for  the  Individual 

Collaborative  Capacity  Items  and  Scale 


Individual  Collaborative  Capacity  Items 

Item 

Mean 

Item 

Standard 

Deviation 

Our  employees  have  the  collaborative  skills  (e.g.,  conflict 
management  and  team  process  skills)  needed  to  work  effectively 
with  other  agencies. 

4.0 

1.3 

Members  of  my  organization  are  aware  of  the  capabilities  of  other 
organizations  with  which  we  have  to  work. 

3.9 

1.2 

Members  of  my  organization  respect  the  expertise  of  those  in  other 
organizations  with  whom  we  work. 

4.5 

1.2 

Members  of  my  organization  understand  how  our  work  relates  to 
the  work  of  other  organizations  with  whom  we  need  to  collaborate. 

4.3 

1.2 

Members  of  my  organization  are  able  to  appreciate  another 
organization’s  perspective  on  a  problem  or  course  of  action. 

4.0 

1.1 

Members  of  my  organization  are  willing  to  engage  in  a  shared 
decision-making  process  with  other  organizations  when  addressing 
interorganizational  issues. 

4.2 

1.2 

People  in  my  organization  seek  input  from  other  organizations. 

4.1 

1.3 

Individual  Collaborative  Capacity  Scale  Statistics 

Mean 

Standard  Deviation  Sample  Size  Coefficient  Alpha 

4.1 

1.2  258  .86 
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Barriers  to  Collaboration.  All  the  items  and  scales  described  to  this  point  have 
been  facilitating  factors  or  enablers  that  support  the  development  and  maintenance  of 
collaborative  capacity.  In  contrast,  the  items  in  Table  12  are  barriers  to  collaboration. 
Agreement  on  these  items  is  undesirable,  whereas  agreement  on  other  items  is 
desirable.  This  requires  that  the  item  means  and  the  scale  mean  must  be  reverse 
coded  to  compare  it  to  other  scales,  subtracting  it  from  7  so  that  a  mean  of  3.3  becomes 
3.7.  This  reversal  is  used  in  comparisons  among  scales  in  subsequent  sections  of  this 
report. 


Five  somewhat  diverse  items  are  related  to  each  other  as  Barriers  to 
Collaboration  with  a  good  correlation  of  .75.  The  items  are  diverse;  they  assess — from 
top  to  bottom  in  the  Table  12 — aspects  of  history,  individual  collaborative  capacity,  role 
conflict,  policies,  and  unique  requirements.  The  items  cover  a  number  of  domains 
within  the  collaborative  capacity  model  and  cannot  be  placed  within  any  single  domain. 

Table  12.  Means,  Standard  Deviations,  and  Coefficient  Alpha  for  the  Barriers  to 

Collaboration  Items  and  Scale 


Barriers  to  Collaboration  Items 

Item 

Mean 

Item 

Standard 

Deviation 

A  history  of  interorganizational  conflict  affects  our 
interorganizational  capability. 

3.6 

1.5 

People  in  my  organization  tend  to  be  suspicious  and  distrustful  of 
their  counterparts  in  other  organizations. 

3.1 

1.4 

1  face  incompatible  requirements  or  requests  when  working  with 
other  organizations. 

3.3 

1.1 

Conflicting  organizational  policies  make  collaboration  difficult. 

3.6 

1.3 

My  organization’s  unique  requirements  make  collaboration  difficult. 

2.7 

1.5 

Barriers  to  Collaboration  Scale  Statistics 

Mean 

Standard  Deviation 

Sample  Size 

Coefficient  Alpha 

3.3 

1.4 

136 

.75 
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Interagency  Teams.  Respondents  in  the  Homeland  Defense  and  Security 
sample  were  directed  to  the  items  in  Table  13  if  they  answered  “yes”  to:  “My 
organization  has  a  representative  on  an  interorganizational  team."  Respondents  in  the 
Acquisition  and  Contracting  sample  only  were  directed  to  these  items  if  they  indicated 
they  had  served  on  one  or  more  “interorganizational  special  project  or  tiger  teams.” 

The  items  might  be  regarded  as  assessing  an  aspect  of  the  domain  of  lateral 
processes.  Task  forces  are  a  common  form  of  lateral  coordination  between 
differentiated  units  that  face  a  common  problem  space  where  collaboration  is  required 
(Galbraith,  2002).  These  items  have  very  good  to  excellent  reliability. 

Table  13.  Means,  Standard  Deviations,  and  Coefficient  Alpha  for  the  Interagency 

Team  Items  and  Scale 


Interagency  Team  Items 

Item 

Item 

Mean 

Standard 

Deviation 

My  organization  gives  members  of  special  project  teams  (or  tiger 
teams)  adequate  authority  to  speak  on  behalf  of  the  organization.1 

4.2 

1.4 

My  organization  supports  the  decisions  and  recommendations  of 
the  special  project  or  tiger  team.2 

4.3 

1.2 

Interagency  Team  Scale  Statistics 

Mean  Standard  Deviation  Sample  Size 

Coefficient  Alpha 

4.2  1.3  193 

.85 

1  The  Homeland  Defense  and  Security  sample  item  did  not  include  the  words  “project  teams  (or  tiger 
teams);  it  only  referred  to  “members  of  the  interorganizational  team.” 


2  The  Homeland  Defense  and  Security  sample  item  did  not  include  the  words  “the  project  or  tiger  team”;  it 
referred  to  “the  interorganizational  team.” 
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The  Interorganizational  Collaborative  Capacity  Scales.  Table  14  repeats  the 
scale  statistics  presented  in  Tables  2  through  13,  ranking  them  from  high  to  low.  As  a 
reminder,  all  items  used  a  6-point  Likert  scale  ranging  from  “Strongly  Disagree”  to 
“Strongly  Agree.”  Thus,  3.5  is  the  scale  midpoint.  Values  higher  than  3.5  indicate 
agreement,  and  values  of  6  indicate  unanimous  agreement.  In  addition,  the  reader 
should  note  that  the  Barriers  to  Collaboration  scale’s  mean  has  been  reverse  coded  to 
enable  comparison  with  other  scales.  Again,  this  is  because  it  is  the  only  scale  where  a 
high  value  indicates  a  lack  of  collaborative  capacity.  (Again,  reverse  coding  the  mean 
for  a  6-point  scale  is  determined  by  subtracting  the  mean  from  7).  Because  the  scale 
has  been  reversed,  it  is  labeled  Lack  of  Barriers  to  Collaboration. 

From  discussions  with  our  students,  many  of  whom  are  clearly  SMEs,  we  know 
that  practitioners  can  engage  in  lengthy  conversations  about  the  pattern  of  results  they 
see  in  the  items  that  make  up  the  scales  and  in  the  pattern  of  results  in  the  scales 
themselves.  A  few  examples  are  presented  here  to  illustrate  scale  interpretation. 

In  various  administrations,  the  Need  to  Collaborate  scale  and  its  items  are  the 
highest  in  the  survey.  This  indicates  that  the  respondents  report  that  their  organization 
sees  the  need  to  collaborate.  This  will  perhaps  remain  a  common  result.  We  have 
selected  contexts  in  Acquisition  and  Contracting  and  in  Homeland  Defense  and  Security 
in  which  messages  from  the  highest  levels  of  the  chain  of  command  and  even  stories  in 
the  national  media  emphasize  the  importance  and  criticality  of  collaboration.  In 
addition,  commands  that  support  administering  the  survey  would  be  unlikely  to  do  so 
unless  they  perceived  collaboration  to  be  an  important  priority. 

The  Resource  Investments  mean  is  considerably  lower  than  the  Felt  Need  mean 
and  somewhat  lower  than  the  Strategic  Collaboration  mean.  Students  often  discuss 
these  items  in  terms  of  the  challenges  of  “talking  the  talk”  versus  “walking  the  walk”  (cf. 
Argyris,  1992/1974). 

The  Structural  Flexibility  scale  reveals  more  flexibility  than  some  might  expect, 
given  that  these  respondents  generally  work  for  large  government  bureaucracies.  The 
standard  deviation,  as  for  the  other  scales,  indicates  a  range  of  values.  Some 
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individuals  perceive  more  flexibility  than  others,  and  determining  the  sources  of  the 
distribution  of  this  flexibility-inflexibility  throughout  the  organization  (i.e.,  running  cross¬ 
tabulations  for  different  units)  could  lead  to  insights. 

Reward  Systems  shows  a  relatively  low  score,  lower  than  Structural  Flexibility. 
Some  practitioners  note  that  it  is  easier  to  reorganize  and  change  procedures  than  to 
change  institutionalized  reward  systems.  Because  incentives  and  rewards  are 
especially  powerful  for  generating  organizational  culture,  discussions  in  which  people 
explain  how  reward  systems  often  fail  to  support  collaborative  activities  represent  a 
potentially  powerful  opportunity  for  action  strategies. 

Metrics  for  Collaboration  is  rather  consistently  among  the  lowest  scores,  and  the 
low  values  promote  interesting  discussions.  These  include  the  issue  of  how  important 
such  measures  really  are,  the  degree  to  which  such  measures  might  produce 
counterproductive  behaviors,  and  the  role  of  leadership  in  assessing  individual  and 
motivating  performance.  Individuals  sometimes  note  that  the  ICC  survey  is  itself  an 
attempt  to  develop  collaboration  metrics. 

The  surveys  obviously  measure  respondents’  perceptions,  and  perceptions  tend 
to  differ,  sometimes  being  accurate  and  sometimes  being  inaccurate4.  To  give  one 
example,  consider  the  scale  that  assesses  strategic  collaboration.  It  is  possible  that  low 
scores  represent  a  realistic  assessment  of  leader  behavior.  Leadership  may  not  be 
oriented  toward  ICC,  and  they  may  be  failing  to  address  fundamental  issues  of  goal 
setting  and  failing  to  meet  with  leaders  of  other  organizations.  However,  it  is  also 
possible  that  leadership  is  doing  these  things,  but  there  is  a  communication  gap,  and 
the  actions  being  taken  are  not  being  perceived  by  all  or  some  segments  within  the 
organization.  Analyzing  the  different  responses  of  departments  within  an  organization 
and  of  different  levels  of  the  hierarchy  provide  insights  about  respondents’  perceptions. 
Whether  the  results  represent  a  reality  of  leader  behavior  or  misperceptions,  or  some 


4  Perceptions  also  shape  reality,  which  is  another  reason  that  collecting  survey  data  can  be  useful:  they 
provide  insights  into  individual  expectations,  orientations,  and  behavior. 
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combination,  action  plan  strategies  can  be  generated  to  improve  this  domain  of 
collaborative  capacity. 


Table  14.  Means,  Standard  Deviations  (S.D.),  Sample  Size  (n)  and  Coefficient 
Alpha  for  the  Collaborative  Capacity  Scales 


Scale 

Mean 

S.D. 

n 

Coefficient 

Alpha 

Need  to  Collaborate 

4.7 

1.3 

307 

.81 

Strategic  Collaboration 

4.3 

1.4 

251 

.85 

Social  Capital 

4.2 

1.3 

307 

.79 

Interagency  Team 

4.2 

1.3 

193 

.85 

Structural  Flexibility 

4.1 

1.2 

135 

.78 

Information  Sharing 

4.1 

1.4 

226 

.83 

Individual  Collaborative  Capacity 

4.1 

1.2 

258 

.86 

Resource  Investments 

3.7 

1.4 

227 

.88 

Lack  of  Barriers  to  Collaboration1 

(3.7)1 

1.4 

136 

.75 

Collaborative  Learning 

3.5 

1.4 

225 

.85 

Reward  Systems 

3.4 

1.5 

268 

.86 

Metrics  for  Collaboration 

3.0 

1.5 

264 

.83 

collaborative  capacity.  It  is  thus  reversed  so  that  it  can  be  compared  to  the  other  scales  and  relabeled  as 
Lack  of  Barriers  to  Collaboration. 


Comparing  the  ICC  of  a  Set  or  Community  of  Organizations 

The  ICC  results  can  be  used  to  compare  samples  from  different  organizational 
sets,  different  communities  of  practice,  or  even  different  industries.  By  administering 
surveys  to  individuals  from  multiple  organizations,  we  should  be  able  to  describe  the 
collaborative  capacity  of  that  organizational  set.  Table  15  compares  the  means  of  two 
such  sets:  the  Homeland  Defense  and  Security  (HDS)  and  Acquisition  and  Contracting 
(AC)  samples. 
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Table  15.  Means,  Standard  Deviation  (S.D.),  and  Sample  Sizes  (n)  for  Homeland 
Defense  &  Security  and  for  Acquisition  &  Contracting  Communities 


Scale _ 

Need  to  Collaborate 
Strategic  Collaboration 
Resource  Investments 
Structural  Flexibility 
Reward  Systems 
Metrics  for  Collaboration 
Social  Capital 
Information  Sharing 
Collaborative  Learning 
Individual  Collaborative 
Capacity 

Lack  of  Barriers  to 
Collaboration1 
Interagency  Team _ 


Homeland  Defense  &  Acquisition  & 

_ Security _ Contracting 


Mean 

S.D. 

n 

Mean 

S.D. 

n 

5.0 

1.0 

145 

4.3 

1.2 

49 

4.4 

1.0 

145 

3.8 

1.3 

49 

4.0 

1.2 

144 

3.4 

1.2 

49 

4.1 

1.0 

145 

4.1 

1.1 

49 

3.4 

1.3 

145 

3.1 

1.2 

49 

2.9 

1.2 

141 

2.8 

1.2 

49 

4.5 

1.1 

144 

3.9 

1.2 

49 

4.2 

1.2 

145 

3.6 

1.1 

49 

3.7 

1.1 

145 

2.9 

1.0 

49 

4.2 

1.0 

144 

3.9 

1.0 

49 

(3.7)1 

1.0 

145 

(3.6)1 

.9 

49 

4.6 

1.1 

117 

3.5 

1.3 

48 

1  The  Barriers  to  Collaboration  scale  is  the  only  scale  in  which  a  higher  value  represents  a  lower 
collaborative  capacity.  It  is  thus  reversed  so  that  it  can  be  compared  to  the  other  scales  and  relabeled  as 
Lack  of  Barriers  to  Collaboration. 


Profiles  and  Interpretations.  The  data  in  Table15  can  also  be  displayed  using 
line  graphs;  such  graphs  often  highlight  patterns  and  results  that  are  less  obvious  in 
Tables,  although  they  risk  overemphasizing  the  central  tendency  measure  at  the 
expense  of  an  assessment  of  variability.  The  means  in  Figure  3  would  seem  to  indicate 
no  significant  difference  between  organizational  communities  in  terms  of  Structural 
Flexibility,  Reward  Systems  and  Metrics  for  Collaboration,  Individual  Collaboration,  and 
Barriers  to  Collaboration.  The  main  differences  appear  to  be  that  the  Homeland 
Defense  and  Security  sample’s  means  are  somewhat  higher  in  the  strategic  domain 
comprising  Need  to  Collaborate,  Strategic  Collaboration,  and  Resource  Investments 
and  in  the  lateral  processes  domain  of  Social  Capital,  Information  Sharing,  and 
Collaborative  Learning.  (The  Interagency  Team  scale’s  high  value  in  the  HDS  sample 
is  interesting  as  it  might  be  viewed  as  a  combination  of  strategic  leadership  and  lateral 
processes.) 
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6 


Mean 

Standard 

Deviation 


Mean 

Standard 

Deviation 


Homeland  Defense  and  Security  Means  and  Standard  Deviations 


5.0 

4.4 

4.0 

4.1 

3.4 

2.9 

4.5 

4.2 

3.7 

4.2 

(3.7)* 

4.6 

Mean 

1.0 

1.0 

1.2 

1.0 

1.3 

1.2 

1.1 

1.2 

1.1 

1.0 

1.0 

1.1 

Standard 

Deviation 

Acquisition  and  Contracting  Means  and  Standard  Deviations 

4.3 

3.8 

3.4 

4.1 

3.1 

2.8 

3.9 

3.6 

2.9 

3.9 

(3.6)* 

3.5 

Mean 

1.2 

1.3 

1.2 

1.1 

1.2 

1.2 

1.2 

1.1 

1.1 

1.1 

0.9 

1.3 

Standard 

1  The  Barriers  to  Collaboration  scale  is  the  only  scale  in  which  a  higher  value  represents  a  lower 
collaborative  capacity.  It  is  thus  reversed  so  that  it  can  be  compared  to  the  other  scales  and  relabeled  as 
Lack  of  Barriers  to  Collaboration. 

Figure  3.  Profiles  with  Means  and  Standard  Deviations  for  the  Homeland  Defense 
&  Security  and  the  Acquisition  and  Contracting  Samples. 

Caution  is  required  in  making  attributions  about  differences  in  organizations  as 
diverse  as  HDS  versus  AC.  To  generalize  about  mean  differences,  comparable 
sampling  procedures  for  the  two  communities  are  required,  which  we  certainly  do  not 
have.  Even  with  excellent  sampling,  the  contextual  frames  of  reference  used  by 
respondents  as  they  take  the  survey  need  to  be  considered  when  interpreting  these 
differences. 

Here  are  a  few  of  the  contextual  differences  characterizing  the  individuals  who 
are  in  the  AC  versus  HDS  samples. 

■  AC  is  relatively  mature  and  institutionalized,  and  the  organizations  have 
been  created  and  mandated  to  facilitate  and  oversee  interorganizational 
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performance.  By  contrast,  much  of  HDS  has  been  recently  reorganized 
and  reconfigured  to  improve  efficiency  and  collaboration,  but  there  are  still 
a  large  minority  of  respondents  who  report  they  do  not  have  a  mandate  to 
collaborate  with  other  organizations,  so  collaboration  depends  on  local 
initiatives. 

■  AC  comprises  many  functionally-focused,  matrixed  organizations  with 
inherently  conflicting  intraorganizational  goals  and  resulting  personal  role 
conflict.  The  functional  areas  have  divergent  cultures  and  goals;  — for 
example  Program  Managers  (PMs)  definitely  have  different  goals  than 
contracting  officers  or  even  systems  engineers.  By  contrast,  HDS 
comprises  independent  organizations  with  distinct  chains  of  command. 

■  AC  comprises  primarily  civil  service  members  of  DoD’s  acquisition 
workforce.  HDS  includes  a  small  minority  of  military  officers  and  some 
federal  civil  servants  from  diverse  agencies,  as  well  as  employees  of 
state,  county,  and  city  governments. 

■  Our  DHS  respondents’  perceptions  of  the  consequences  or  risk  in  the  face 
of  a  failure  of  interagency  collaboration  were  significantly  higher  than  the 
perception  of  risk  in  our  AC  samples. 

Interpretive  Norms.  This  discussion  of  comparing  survey  results  on 
organizational  capabilities  raises  the  question  of  interpretive  norms.  Norms  are 
“descriptive  statistics  that  are  compiled  to  permit  the  comparison  of  a  particular  score 
(or  mean)  with  the  scores  (or  means)  earned  by  the  members  (or  groups  of  members) 
of  some  defined  population  [...].”  (Thorndike,  1971 ,  p.  533).  Norms  in  this  context 
would  allow  an  organization  to  understand  its  relative  standing  on  a  scale  or  profile  of 
scales. 


The  concept  of  norms  for  organizations,  in  contrast  to  norms  for  people,  is 
problematic.  Norms  can  be  developed  and  their  meaning  understood  relatively  easily 
for  individual  human  beings.  However,  developing  norms  for  aggregates  of  people  in 
organizations  is  far  more  difficult  and  ambiguous.  Put  differently,  the  meaning  of  an 
“individual  person”  is  more  uniform  than  of  an  “individual  organization”.  Organizations 
differ  vastly  in  size,  age,  history,  mission,  technology,  member  demographics  and  other 
variables.  In  addition,  organizations  are  nested  hierarchically  within  other  organizations, 
and  sometimes  they  are  matrixed;  their  boundaries  are  more  ambiguous.  Thus,  the 
entire  concept  of  individual  vs.  organizational  norms  requires  rethinking.  However, 


-  29  - 


without  specifically  addressing  these  challenges,  we  argue  that  a  wide  and  continuing 
sampling  of  individuals  from  particular  classes  of  organizations  (e.g.,  HDS  or  AC)  may 
prove  useful  for  comparative  purposes.  There  are  some  commonalities  within  various 
organizational  sets  and  communities,  and  leadership  can  derive  some  sense  of  their 
organization’s  relative  standing  by  cautiously  comparing  their  results  to  the  results  of 
others.  By  increasing  the  sample  size  and  more  rigorously  collecting  data  across  the 
acquisition  community,  it  is  possible  to  generate  information  akin  to  norms.  We 
illustrate  the  potential  value  of  this  in  the  next  section. 

Assessing  One  Organization’s  Interorganizational  Collaborative 
Capacity 

The  previous  sections  of  the  paper  used  a  diverse  set  of  individuals  to  analyze 
the  structure  of  the  ICC  survey  and  assess  its  internal-consistency  reliability  and 
convergent  validity.  It  then  briefly  presented  results  from  two  samples  representing  two 
organizational  sets.  It  also  raised  the  issues  of  context  and  norms  for  organizational 
surveys.  In  this  section,  we  use  the  survey  to  do  a  brief,  summary  assessment  of  two 
units  within  a  much  larger  organizational  system. 

Table  16  shows  the  results  for  two  units  of  a  major  DoD  Contracting 
Administration  organization  that  functions  in  an  interorganizational  context  (cf. 
Kirschman  &  Laporte,  2008).  In  the  following  discussion  of  survey  results,  we  quote 
leaders  associated  with  these  units.  The  quotations  come  from  a  teleconference 
interview  conducted  with  the  top  leadership  team  of  these  units  on  June  22,  2007  and 
from  comments  made  when  the  leadership  of  one  unit  conducted  a  briefing  for  an  NPS 
Acquisition  and  Contracting  class  on  August  6,  2008.  The  organizations  and  their 
managers  are  quoted  anonymously  to  maintain  confidentiality. 

The  organization’s  mission  is  to  enhance  the  integration  of  acquisition  processes 
between  DoD  clients  and  contractors,  generating  cost  savings,  increasing  process 
innovations,  and  ensuring  compliance  to  standards  and  regulations  in  federal  contracts. 
Prior  to  the  survey  administration,  one  manager  (personal  communication,  June  22, 
2007)  said,  “We  are  very  much  a  customer  support  organization.  And  so  that  really 


-  30  - 


requires  that  in  order  to  provide  the  support  that  our  customers  need,  we’ve  got  to 
interact  and  collaborate  with  them  ....  We’ve  got  to  sit  down  with  our  customers  and 
agree  and  talk  and  discuss  what’s  important  to  them.”  In  this  context,  customers  of  the 
organization  are  other  government  agencies;  they  support  these  agencies  with  respect 
to  contractors,  typically  involving  major  programs.  The  leadership  team  appeared  to  be 
dedicating  considerable  time  and  energy  to  generate  and  sustain  interorganizational 
collaborative  capacity.  Indeed,  they  seemed  to  support  administration  of  the  survey 
because  of  their  commitment  to  developing  high  collaborative  capacity  in  their  units. 

Table  16.  Means,  Standard  Deviation  (S.D.),  and  Sample  Sizes  (n)  for  a  Major  DoD 

Contracting  Administration  Organization 


Scale 

Mean 

S.D. 

n 

Need  to  Collaborate 

4.5 

1.1 

46 

Strategic  Collaboration 

4.2 

1.2 

46 

Resource  Investments  in  Collaboration 

3.4 

1.5 

43 

Structural  Flexibility 

4.0 

1.0 

46 

Reward  Systems 

3.8 

1.2 

45 

Metrics  for  Collaboration 

3.6 

1.4 

43 

Social  Capital 

4.0 

1.2 

46 

Information  Sharing 

3.9 

1.2 

46 

Collaborative  Learning 

3.2 

1.3 

46 

Individual  Collaborative  Capacity 

3.9 

1.1 

46 

Lack  of  Barriers  to  Collaboration1 

(3.2)1 

1.1 

46 

Interagency  Team 

3.9 

1.2 

40 

1  The  Barriers  to  Collaboration  scale  is  the  only  scale  in  which  a  higher  value  represents  a  lower 
collaborative  capacity.  It  is  thus  reversed  so  that  it  can  be  compared  to  the  other  scales  and  relabeled  as 
Lack  of  Barriers  to  Collaboration. 

The  organization  has  a  history  of  internal  consolidation,  becoming  more  of  a  joint, 
DoD  based  organization;  thus,  in  the  past  intraorganizational  collaboration  has  been  an 
issue.  Executing  their  mission  requires  “selling”  organizational  customers  on  the 
support  they  can  provide.  In  order  to  be  more  effective  with  customers,  the  organization 
has  gone  through  what  one  manager  (personal  communication,  June  22,  2007)  called  a 
“realignment  that  puts  us  in  a  position  where  we  are  very  focused  on  specific 
customers.” 
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1  The  Barriers  to  Collaboration  scale  is  the  only  scale  in  which  a  higher  value  represents  a  lower 
collaborative  capacity.  It  is  thus  reversed  so  that  it  can  be  compared  to  the  other  scales  and  relabeled  as 
Lack  of  Barriers  to  Collaboration. 

Figure  4.  Profiles  with  Means  and  Standard  Deviations  for  a  Major  DoD 
Contracting  Administration  Organization. 

Figure  4  presents  the  results  from  Table  16  in  the  form  of  a  line  graph.  In 
general,  individuals  in  this  organization  express  mild  or  modest  agreement  on  most 
factors.  There  is,  on  average,  mild  disagreement  regarding  the  adequacy  of  Resource 
Investments  and  engagement  in  Collaborative  Learning,  and  there  is  mild  agreement 
that  there  are  Barriers  to  ICC.  The  standard  deviations  for  two  scales — Resource 
Investments  and  Metrics  for  Collaboration — appear  relatively  high,  indicating  less 
consensus  among  the  organization’s  members  on  these  dimensions. 

Figure  5  provides  additional  perspective  of  these  results.  It  adds  the  results  on 
the  Acquisition  and  Contracting  organizations  described  in  the  previous  section,  and 
these  then  can  serve  as  a  surrogate  for  comparative  norms.  Again,  we  should  be 
cautious  in  our  interpretations,  but  this  comparison  suggests  a  somewhat  more  positive 
perspective  than  we  gain  only  by  viewing  the  organizational  profile  in  isolation.  The 
pattern  of  comparisons  shows  the  Contracting  Administration  organization  as  being 
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equal  or  higher  than  the  normative  sample  (drawn  from  49  different  organizations)  on  all 
the  enabling  factors  and  shows  higher  ratings  in  the  factors  of  Resource  Investments, 
Reward  Systems,  and  Metrics  for  Collaboration.  This  example  should  be  suggestive  of 
the  interpretive  value  of  having  a  larger,  more  representative  data  set  for  Acquisition 
and  Contracting  organizations.  In  the  next  section,  we  discuss  the  subsystem  domains 
and  their  factors,  which  are  illustrated  in  Figure  6. 
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The  Barriers  to  Collaboration  scale  is  the  only  scale  in  which  a  higher  value  represents  a  lower  collaborative 
capacity.  It  is  thus  reversed  so  that  it  can  be  compared  to  the  other  scales  and  relabeled  as  Lack  of  Barriers 
to  Collaboration. 


Figure  5.  Profiles  with  Means  and  Standard  Deviations  for  Major  DoD  Contracting 
Administration  Organization  and  an  NPS  Acquisition  and  Contracting  Sample 

Representing  an  Organizational  Set. 
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-  Purpose  &  Strategy 


■  Need  to  Collaborate  (4.5) 

■  Strategic  Collaboration  (4.2)' 

■  Resource  Investments  (3.4) 


1  Metrics  (3.6) 

Reward  Systems 

Incentives  &  Reward 
Systems  (3.8) 


-  Structure 


Behavior 
&  Processes. 

Culture 


Lack  of  Barriers 

to  Collaboration  (3.2*) 


-  Lateral  Processes 
■  Social  Capital  (4.0) 


Information  Sharing  (3.9) 
Collaborative  Learning  (3.2) 


People 


Structural  Flexibility  (4.0) 


Individual  Collaborative 
Capacity  (3.9) 


■  Interagency  Team  Support  (3.9) 


Figure  6.  Interorganizational  Collaborative  Capacity  Scale  Values  Organized  by 

Organizational  Domain  for  a  Major  Department  of  Defense  Contracting 

Administration  Organization. 

The  Strategic  Domain.  In  terms  of  the  strategic  domain,  the  Need  to 
Collaborate  scale  and  the  Strategic  Collaboration  scales  have  relatively  high  values, 
with  means  of  4.3  and  3.8  indicating  moderate  agreement.  This  is  congruent  with  the 
expressed  collaborative  vision  and  statements  of  those  in  the  top  leadership  team  to 
which  the  organization  reports.  A  deputy  (personal  communication,  June  22,  2007) 
noted  that,  a  “key  point”  for  his  organization  was  “bringing  in  the  needed  stakeholders, 
identifying  what  the  key  issues  are,  what  the  common  ground  is  and  then  understanding 
what  we  can  do  to  complement  each  other.”  The  survey  provides  some  evidence  of  the 
degree  to  which  the  message  behind  this  vision  is  accepted. 

In  our  teleconference  with  the  top  leadership  team,  we  clearly  heard  collaboration 
and  improved  collaboration  with  customers  and  contractors  as  strategic  priorities.  But 
the  timeline  suggested  that  the  prime  movement  from  the  highest  levels  has  come  in  the 
past  two  years.  Clearly,  getting  the  organizational  domains  illustrated  in  Figure  6  into  a 
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new  realignment  around  improved  collaboration  in  the  context  of  the  federal 
bureaucracy  is  likely  to  be  a  process  that  must  overcome  considerable  inertia. 

The  Resource  Investments  scale,  although  higher  than  the  “normative”  sample 
result,  remains  low,  with  a  mean  of  3.4,  which  suggests  this  may  be  a  barrier  to 
collaboration  (and  is  certainly  perceived  as  such  by  many  in  the  organization).  It  was 
also  raised  as  a  primary  issue  or  barrier  by  top  leadership: 

One  of  the  things  that  come  to  mind,  just  right  away,  in  barriers,  is  a 
resource  situation.  And  having  the  right  talent  ...  to  be  able  to  provide  the 
support  our  customers  need.  This  agency  has  gone  down  to — I’ve  been 
here  less  than  two  years — so  I  have  the  20,000  number — we  are  roughly  a 
10,000  person  agency  now.  So  our  resources  have  gone  down 
significantly,  (personal  communication,  June  22,  2007) 

Indeed,  a  commander  of  one  of  the  units  (personal  communication,  August  6,  2008) 
said  he  simply  lacked  the  personnel  and  other  resources  to  engage  all  the  tasks  so  that 
a  key  part  of  his  job  involved  risk  management  to  assess  where  slippage  was  least 
problematic.  Thus  the  strategic  picture  with  respect  to  collaborative  capacity  is  trying  to 
do  more  with  less,  so  the  theme  of  efficiency  becomes  paramount. 

The  Structural  Domain.  The  organization  is  the  result  of  a  restructuring  that 
incorporated  service  organizations  into  a  joint  entity.  This  was  motivated  primarily  by 
the  need  for  greater  efficiency.  It  allows  private  contractors  building  products  for 
multiple  services  to  simplify  their  interface  problems  with  the  DoD  bureaucracy.  More 
recently,  the  organization  has  reorganized  from  a  geographic  divisional  structure  to  a 
more  product-oriented  structure.  These  changes  in  departmentation  may  well  affect 
personnel’s  perceptions  of  flexibility.  However,  it  remains  a  bureaucracy  with  some 
regulatory  responsibilities. 

The  organization’s  previous  structural  inflexibility  was  described  in  a  PowerPoint 
brief  presented  to  students  at  the  Naval  Postgraduate  School  (personal  communication, 
August  6,  2008):  “Someone  above  writes  the  rules — workers  follow  the  rules — masters 
check  to  see  that  workers  are  following  the  rules  and  if  they  are,  victory  is  proclaimed  on 
all  fronts.”  Management  rejected  this  rule-driven  behavioral  legacy,  calling  it  “too 
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internally  focused”  and  emphasizing  more  responsiveness  to  customers.  Leadership  is 
clearly  aware  that  improving  structural  flexibility  in  the  interest  of  better  customer  service 
is  a  long-term  project.  The  Structural  Flexibility  mean  of  4.0  indicates  some  agreement 
that  there  is  some  flexibility  in  the  structural  domain.  This  is  a  more  positive  result  than 
might  be  expected  if  the  history  of  rule-driven  behavior  described  above  was  still 
accurate. 

The  Reward  Systems  Domain  and  Metrics.  In  the  ICC  model,  metrics  are 
viewed  as  overlapping  two  domains:  reward  systems  and  strategy.  Metrics  should 
relate  to  goals  and  objectives,  but  if  metrics  are  used  for  individual  and  unit 
accountability,  then  they  become  integrated  into  the  reward  system.  Leadership  in  the 
Contracting  Administration  organization  had  invested  considerable  time,  energy  and 
resources  in  developing  metrics  and  linking  them  to  performance.  Consider  the 
following  from  two  high  level  leaders  in  a  teleconference  (personal  communication,  June 
22,  2008): 

We  have  metrics,  and  we  have  a  performance  commitment  that  we  set 
down  with  our  customers,  and  we’ve  agreed  that  this  is  important  and  this 
is  what  we  are  going  to  do  with  you.  And  there’s  a  way  to  grade  those, 
and  that’s  set  out  . . .  and  as  a  result  of  that,  then  we  know  . . .  where  we  fall 
short. 

Information  is  put  into  an  agency  tool  that  provides  agency  level  visibility 
from  the  agency  director  down  to  the  actual  employee  implementing  the 
strategy. 

In  most  of  our  workshops,  metrics  has  been  the  lowest  scale  score.  The 
Metrics  for  Collaboration  scale’s  mean  value  of  3.6  is  no  doubt  less  than  the 
management  team  would  like  to  see,  but  a  result  on  this  scale  that  is  even 
slightly  into  the  agree  range  appears  promising  to  us. 

Our  conversations  with  leadership  about  their  efforts  to  reward  and  recognize 
individuals  were  made  in  the  context  of  an  understaffed  organization  with  a  mission  of 
promoting  interorganizational  collaboration.  The  Reward  Systems  scale  mean  of  3.8 
suggests  positive  results  in  maintaining  focus  in  difficult  conditions. 
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The  People  Domain — Individual  Collaborative  Capacity.  Leadership 
(personal  communication,  June  22,  2007)  reported  that  they  sometimes  lacked  people 
with  the  required  technical  skill  sets  (e.g.,  engineering  talent  and  software  skills).  They 
also  asserted  the  importance  of  “critical  thinking  skills”  and  “soft  skills”  for  dealing  with 
conflict,  explaining  resource  constraints  and  limitations  on  support  that  can  be  offered. 
However,  they  did  not  express  any  sense  that  the  people  in  their  organization  were 
deficient  in  such  skills;  they  did  not  regard  it  as  a  problematic  barrier  to  collaboration. 
They  did  indicate  that  there  was  “always  room  for  improvement”  with  such  soft, 
collaborative  skills.  The  discussion  was  consistent  with  the  Individual  Collaborative 
Capacity  scale  mean  of  3.9;  qenerally  positive  with  room  for  improvement. 

The  Lateral  Processes  Domain.  The  Social  Capital  mean  of  4.0  is  somewhat 
positive,  revealing  moderate  agreement  with  items  indicating  individuals  take  the 
initiative  and  know  who  to  contact  in  other  organizations.  Information  Sharing  is  also 
somewhat  positive,  with  a  mean  of  3.9.  Collaborative  Learning  would  seem  to  be  a 
more  demanding  level  of  lateral  integration,  and  its  lower  mean  of  3.2  is  not  surprising. 
The  organization  exists  in  a  complex  system  of  partners  with  other  organizations,  and 
knowing  who  to  contact  and  developing  effective  lateral  processes  is  itself  a  complex 
endeavor. 

Lack  of  Barriers  to  Collaboration.  Individuals  within  the  DoD  Contracting 
Administration  organization  still  perceive  barriers  to  collaboration,  as  the  relatively  low 
reversed  mean  of  3.2  suggests.  This  is  an  indicator  of  the  inertia  that  leadership  must 
overcome  as  they  develop  ICC  in  the  bureaucratic  context. 
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Assessing  Interorganizational  Collaborative  Capacity 


Developing  a  reliable,  potentially  useful  survey  instrument  has  been  the  focus  of 
our  current  research  efforts  in  order  to  address  the  major  stumbling  block  toward 
advancing  the  theory  development  in  collaborative  capacity.  In  addition,  a  reliable,  valid 
survey  can  be  deployed  using  well-established  methods  of  survey-guided  organizational 
development  (Thomas,  Hocevar,  Jansen,  &  Rendon,  2007.)  In  this  last  section,  we 
would  like  to  put  the  survey  in  a  somewhat  broader  perspective,  discussing  the  larger 
assessment  process  in  which  the  survey  is  used,  the  context  dependence  of  survey 
scores,  the  issues  of  leading  and  lagging  factors  in  open  systems  models  as  they  relate 
to  profiles  of  score  values,  and  continuing  and  future  research  in  ICC. 

Survey  Deployment  and  the  Assessment  ICC 

The  ICC  survey  was  designed  to  fit  into  a  larger  assessment  process.  Assessing 
an  organization’s  collaborative  capacity  ideally  involves  not  only  a  systematic  sampling 
of  members’  perceptions  using  the  diagnostic  survey,  but  also  qualitative  methods,  such 
as  interviews  and  focus  groups.  Interviews,  focus  groups,  and  discussions  with 
management  prior  to  survey  administration  often  results  in  a  more  tailored,  more  useful 
survey.  Post-survey  interviews  and  focus  groups  add  depth  and  understanding  to 
survey  results.  The  survey  was  designed  to  be  used  in  action  research  and 
organizational  development  context,  and  optimal  learning  occurs  when  researchers, 
leadership,  and  organizational  members  collaborate. 

Surveys  have  strengths,  but  they  also  have  limitations.  They  are  an  efficient 
means  of  collecting  data  to  assess  a  wide  landscape,  to  efficiently  understand  the 
structure  and  intensity  of  factors  in  a  larger  sample  of  people.  Surveys  also  signal  that 
management  thinks  a  topic  is  important  enough  to  ask  their  people  to  take  time  from 
other  tasks  in  order  to  provide  feedback.  However,  interviews  and  focus  groups  are 
better  able  to  answer  process  questions  relating  to  causes  and  dynamics  of 
collaborative  capacity;  they  present  a  richer,  deeper  picture  when  deployed.  The  ICC 
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research  program’s  diagnostic  tools  include  a  set  of  interview  questions  to  assess 
context  and  processes.  Appendix  B  provides  a  sample  of  the  interview  questions. 

Context  Dependence  of  Survey  Scores 

Quantitative  survey  results  at  the  micro-level  require  contextual ization  at  the 
macro  level.  Context  has  become  more  important  to  organizational  theorists  and 
researchers  in  understanding  the  situational  and  temporal  boundary  conditions  of 
theories  and  survey  results  (cf.  Bamberger,  2008).  The  structure,  dynamics  and  even 
the  meanings  of  collaboration  and  collaborative  capacity  depend  on  context,  and  a 
theory  of  collaborative  contexts  may  be  required  to  understand  collaborative  capacity. 
(Organizational  demographics  are  assessed  on  the  survey  as  one  means  of  assessing 
context.) 

We  caution  against  reading  too  much  into  any  differences  between  means 
between  the  HDS  and  AC  samples  above,  especially  because  of  different  sampling 
procedures.  However,  if  comparable  sampling  techniques  had  been  applied  to  two 
samples,  caution  would  still  be  required  in  interpreting  differences.  Organizations  differ 
so  greatly  in  terms  of  their  mission  and  tasks,  the  environments  in  which  they  function, 
age  and  maturity,  history,  size,  missions,  work  processes  and  technology,  and  structure, 
that  making  sense  of  results  requires  contextual  knowledge. 

Leading  and  Lagging  Factors  in  the  Open  Systems  Organizational 
Models 

The  ICC  model  is  an  open  systems  model;  it  emphasizes  that  organizations 
depend  on  the  congruence  or  fit  (i.e.,  fitness)  of  their  subsystems  (i.e.,  domains)  with 
respect  to  each  other  and  the  larger  environment.  To  achieve  fitness  in  their  political, 
economic,  social  and  organizational  environments,  organizations  develop  habitual 
routines  and  patterns  of  action.  The  commonality  in  the  pattern  of  ICC  profiles  may  thus 
reflect  a  systemic  state  of  fitness  and  inertia  of  the  public  bureaucracies  in  our  sample. 

Organizational  development  and  change — including  developing  ICC — requires  a 
systemic  approach  if  it  is  to  be  sustained.  However,  trying  to  change  an  organization  in 
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its  entirety  generally  is  not  possible.  Leadership  must  choose  which  subsystem 
domains  they  will  initially  develop  and  which  subsystem  domains  will  be  allowed  to  lag 
and  be  changed  in  the  future  (Kaplan  &  Norton,  1992). 

For  our  organizations,  the  Metrics  for  Collaboration  scale  has  typically  had  the 
lowest — or  one  of  the  lowest — scores.  By  contrast,  Felt  Need  is  often  one  of  the  highest 
scores.  Based  on  our  feedback  sessions  with  our  students  and  other  SMEs,  generating 
a  sense  of  felt  need  is  often  a  lead  factor  in  organizational  change;  managers  often 
begin  organizational  development  by  communicating  a  sense  of  urgency.  Developing  a 
sense  of  felt  need  also  seems  to  be  less  subject  to  inertia  than  changes  involving  other 
subsystem  domains.  By  contrast,  our  feedback  sessions  suggest  that  developing 
metrics  to  assess  collaboration  often  is  a  lagging  developmental  effort;  it  is  often 
delayed  because  of  the  considerable  time  and  skill  required  for  development  and 
implementation.  However,  although  it  might  not  be  typical,  leaders  may  choose  to  use 
metrics  to  lead  change  efforts.  The  DoD  Contracting  Administration  organization 
described  in  this  report  arguably  is  using  a  metrics  management  system  as  a  leading 
factor  in  their  change  efforts.  Using  the  ICC  survey  longitudinally  may  reveal  how 
collaborative  capacity  is  developing  in  the  subsystem  domains  composing  the 
organization. 

Continuing  and  Future  Research 

Feedback  workshops  with  students/SMEs  (Thomas,  Hocevar,  Jansen,  & 

Rendon,  2008)  combined  with  the  item  analysis  and  scale  development  in  this  report 
lend  considerable  confidence  to  the  dimensionality  of  the  ICC  model  and 
operationalizations  reported  here.  In  our  continuing  research,  we  expect  to  generate  a 
much  larger  sample  scale  that  will  provide  an  opportunity  for  more  rigorous  multivariate 
analysis. 

Research  is  currently  underway  to  analyze  how  the  survey  is  used  in  the  context 
of  what  we  term  “relative  collaborative  capacity.”  The  ICC  survey  described  in  this 
report  assesses  an  organization’s  capacity  to  enter  into  relationships  with  unspecified 
partners;  it  assesses  ICC  with  respect  to  generalized  others.  By  contrast,  the  research 
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in  progress  on  “relative  collaborative  capacity”  focuses  on  one  specific  partner;  the 
survey  questions  name  one  specific  organization  and  thus  assess  ICC  for  this  one 
specific  relationship. 

It  is  possible  to  use  the  dimensions  in  the  ICC  model  to  generate  a  performance 
appraisal  of  collaborative  relationships.  In  such  a  case,  items  would  need  to  be 
rewritten  or  other  summary  scales  created  whereby  the  members  of  organizations  judge 
their  partner  organizations  with  respect  to  on-going  relationships.  It  is  even  conceivable 
that  such  measures  could  become  integrated  into  performance  appraisals,  thus  creating 
a  metrics  to  drive  incentives  and  reward  actions  that  develop  and  sustain  ICC. 

Validation  of  the  ICC  survey  requires  research  with  dependent  variables  on 
interorganizational  performance  or  alternative  assessments  of  collaboration  processes 
(e.g.,  nominations  of  extreme  cases  — excellent  collaborators  versus  weak  or 
dysfunctional  collaborators —  by  experts  or  top  leaders).  This  would  provide  an 
external,  empirical  validation  for  the  factors  specified  in  the  ICC  model,  which  at  this 
point  relies  primarily  on  face  validity  and  convergent  validity.  This  is  the  most 
challenging  research  to  perform,  requiring  more  resources  and  management  support 
than  any  previous  efforts. 

Richer  diagnoses  of  an  organization’s  ICC  using  qualitative  methods  in 
conjunction  with  the  survey  also  are  needed.  Understanding  collaborative  capacity 
processes  and  dynamics  requires  qualitative  methods  such  as  interviews  and  focus 
groups  as  well  as  case  studies.  Such  collaborative  efforts  between  researchers  and 
leaders/managers  support  the  “Action  Research”  agenda  of  simultaneously  improving 
organizational  functioning  and  developing  better  theories  and  measurements. 

These  last  two  issues — research  with  dependent  variables  and  qualitatively 
enriched  research  that  is  used  in  conjunction  with  the  survey — are  the  necessary  next 
steps  to  furthering  our  understanding  of  the  dynamics  of  collaborative  capacity  and 
simultaneously  improving  ICC. 
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Conclusion 


At  this  stage  of  our  research,  we  have  considerable  confidence  in  the  usefulness 
of  the  ICC  model  and  the  ICC  survey  scales  reported  here.  The  model  of  collaborative 
capacity  in  Figure  6  conceptualizes  an  organization’s  capability  to  enter  into 
partnerships  as  a  systemic  state.  It  defines  that  state  in  terms  of  a  set  of  factors  (e.g., 
structural  flexibility,  metrics)  organized  into  the  subsystem  domains  of  strategy,  reward 
systems,  structure,  lateral  processes,  and  people  (Galbraith,  2002;  Hocevar,  Thomas,  & 
Jansen,  2006).  This  fits  into  the  themes  and  constructs  identified  in  the  literature  and 
our  own  inductive  research.  We  have  created  scales  to  assess  the  factors  that  define 
the  state  of  ICC,  and  these  possess  internal-consistency  reliability  and  convergent 
validity.  The  items  composing  the  scales  suggest  action  strategies  for  developing 
collaborative  capacity.  Although  continuing  refinement  and  validation  is  necessary,  we 
believe  our  attempts  are  representative  of  the  state  of  the  art  in  theory  and  research  on 
ICC. 


The  problems  facing  the  citizens  of  the  United  States  and  the  people  of  the  world 
are  complex,  interdependent,  dynamic,  and  thus  generate  high  levels  of  uncertainty. 
They  do  not  fit  neatly  into  the  categories  of  academic  disciplines  or  of  single  agencies  or 
organizations.  They  require  collaboration  among  people,  teams,  and  organizations, 
often  in  a  context  of  limited  resources.  They  require  collaboration  to  bring  together 
information,  knowledge  and  expertise  located  in  diverse  organizations.  Collaboration 
across  organizational  boundaries  is  thus  critically  important.  It  also  is  exceptionally 
challenging,  requiring  systematically  assessing  and  developing  capabilities  in  large 
organizational  systems.  The  ICC  survey  is  meant  to  be  another  tool  for  leaders  and 
managers  engaged  in  these  challenging  and  important  efforts. 
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Appendix  B.  Select  Interview  Questions  for 
Assessing  Interorganizational  Collaborative 
Capacity 


What  is  the  central  problem  or  opportunity  that  motivates  efforts  in  collaboration? 

What  do  you  see  as  the  primary  value/outcomes  of  collaboration? 

What  assets — in  terms  of  people,  technology,  or  money — have  been  dedicated  to 
building  collaborative  capacity? 

Are  these  commitments  increasing  or  decreasing  or  remaining  stable? 

What  are  the  risks  if  collaboration  is  not  effective? 

To  what  extent  is  there  consensus  about  the  value  of  collaboration? 

To  what  extent  is  there  consensus  about  how  to  collaborate? 

What  individuals,  groups  or  organizations  have  the  greatest  stake  in  the  success  or 
failure  of  the  collaboration? 

Is  there  an  interagency  team  (i.e. ,  a  formal  coordination  committee  with 
representatives  from  the  different  agencies)? 

Who  are  the  participants? 

What  kinds  of  decisions  are  made  by  the  interagency  or  interorganizational  teams 
(e.g.,  an  interagency  task  force,  special  project  teams,  tiger  teams)? 

How  are  people  directly  affected  by  decisions  made  by  the  interagency  or 
interorganizational  team? 

How  do  your  line  mangers  need  to  interact  with  counterparts  in  your  partner 
organizations? 

How  do  your  first-line  supervisors  and  their  people  have  to  work  together  with  the 
partner  organizations? 

In  what  context? 

To  build  team  relationships  and  “joint”  skills? 

Are  there  formal  liaisons  between  participating  organizations? 

How  do  these  work? 

Do  you  have  full  time  employees/staff  dedicated  to  interagency/interorganizational 
work? 
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